It might be interesting to replace from types import MappingProxyType with built-in frozendict. But currently, the GitHub Action CI runs mypy with Python 3.12 which doesn't have frozendict.

It might be interesting to replace from types import MappingProxyType with built-in frozendict. But currently, the GitHub Action CI runs mypy with Python 3.12 which doesn't have frozendict.

Adding # type: ignore[name-defined] is a quick fix.

This:

diff --git a/Lib/tomllib/_parser.py b/Lib/tomllib/_parser.py
index b59d0f7d54b..96f189537cf 100644
--- a/Lib/tomllib/_parser.py
+++ b/Lib/tomllib/_parser.py
@@ -4,7 +4,7 @@
 
 from __future__ import annotations
 
-from types import MappingProxyType
+__lazy_modules__ = ["tomllib._re"]
 
 from ._re import (
     RE_DATETIME,
@@ -42,7 +42,7 @@
 KEY_INITIAL_CHARS: Final = BARE_KEY_CHARS | frozenset("\"'")
 HEXDIGIT_CHARS: Final = frozenset("abcdef" "ABCDEF" "0123456789")
 
-BASIC_STR_ESCAPE_REPLACEMENTS: Final = MappingProxyType(
+BASIC_STR_ESCAPE_REPLACEMENTS: Final = frozendict(  # type: ignore[name-defined]
     {
         "\\b": "\u0008",  # backspace
         "\\t": "\u0009",  # tab

Gets us from 4ms:

To ~0ms:

I updated the PR to replace types.MappingProxyType with frozendict type thanks to # type: ignore[name-defined] annotation (to please mypy gods).

I ran benchmarks on the latest PR using Python built in release mode (gcc -O3) on Fedora 43:

• According to -X importtime, with this change, import tomllib takes 828 us instead of 9.0 ms on main (10.9x faster).
• Using python -m pyperf command with ./python -S, with this change, import tomllib takes 0.98 ms instead of 9.8 ms (10x faster).

Ok, the PR is now ready for review. cc @hauntsaninja @encukou

I updated the PR to use public names. I also fixed tests for hex/oct/bin numbers.

cc also Tomli maintainer @hukkin.

Hi! 👋

I have an old Tomli branch where I've attempted to do very similar things, but it went into the discard pile, IIRC either because distlib's executable wrapper imports re module so trying to avoid the import didn't help (the situation today is very different because both pip and uv override the executable wrapper), or perhaps it was because the re module was faster at parsing integers than pure Python so the optimization seemed case dependent and controversial. Can't remember exactly 😄

The decimal parsing code I had was mostly as follows. (I've slightly simplified (the original parses underscored decimals too) and commented.)

# If one of these follows a "simple decimal" it could mean that
# the value is actually something else (float, datetime...) so
# optimized parsing should be abandoned.
ILLEGAL_AFTER_SIMPLE_DECIMAL: Final = frozenset(
    "eE."  # decimal
    "xbo"  # hex, bin, oct
    "-"  # datetime
    ":"  # localtime
    "_0123456789"  # complex decimal
)


def try_simple_decimal(src: str, pos: Pos) -> None | tuple[Pos, int]:
    """Parse a "simple" decimal integer.

    An optimization that tries to parse a simple decimal integer
    without underscores. Returns `None` if there's any uncertainty
    on correctness.
    """
    start_pos = pos

    if src.startswith(("+", "-"), pos):
        pos += 1

    if src.startswith("0", pos):
        pos += 1
    elif src.startswith(("1", "2", "3", "4", "5", "6", "7", "8", "9"), pos):
        pos = skip_chars(src, pos, "0123456789")
    else:
        return None

    try:
        next_char = src[pos]
    except IndexError:
        next_char = None
    if next_char in ILLEGAL_AFTER_SIMPLE_DECIMAL:
        return None

    return pos, int(src[start_pos:pos])


def parse_value(
    src: str, pos: Pos, parse_float: ParseFloat, nest_lvl: int
) -> tuple[Pos, Any]:
    ...
    simple_dec_result = try_simple_decimal(src, pos)
    if simple_dec_result is not None:
        return simple_dec_result
    ...

This should do similar things as what you have. One difference is that at least to me ILLEGAL_AFTER_SIMPLE_DECIMAL here seems easier to prove correct (by looking at tomllib's code) than NUMBER_END_CHARS.

I'd personally name the function something other than parse_something simply because no other parse_ function in tomllib returns a None. They all successfully parse or raise an error.

@hukkin: Hi! Oh, it's great that you already explored the "simple decimal number parser" strategy. I really like your implementation, it looks way better than mine! So I simply copy/pasted your code and I added you as a co-author.

I don't know how stdlib tomllib is maintained. Should I contribute this change to https://github.com/hukkin/tomli first? Or is it ok to land such change in the stdlib tomllib module first?

"Tests / CIFuzz / python3-libraries (address)" failed: it generated a TOML file of 1860 characters with with 593 [ array opening character and no ] array closing charracter. tomllib.load() fails with RecursionError. But if I use sys.setrecursionlimit(10_000), tomllib raises tomllib.TOMLDecodeError: Unclosed array (at line 1, column 1553) as expected. So it's a false alarm and I suggest ignoring it for now.